We manually curated 500\(+\) multiple modality PPMI MRI subjects. 5 messages from this effort.
failure rates humans vs processed
we provide scientists handles on this so that can be “strict” or “loose” in data selection
we document these parameters
for algorithm developers
for scientists
Enables investigation of the “meaning” of M3RI at the individual level
Empowers investigation of the “meaning” of M3RI at the population level
Reviewing multiple modality data quality for current phase of collection
we collaborated on cohort definition
first phase - select subjects based on sampling from sites
second phase - select subjects based on population characteristics (MJFF/PPMI)
show table of summary cohort
| p | ||||||
|---|---|---|---|---|---|---|
| (N=47) | (N=1) | (N=4) | (N=230) | (N=217) | ||
| age | 63.5 ± 12.5 | 71.9 ± NA | 59.3 ± 7.1 | 67.7 ± 5.7 | 64.5 ± 9.4 | < 0.001 |
we manually QC’d each modality for each of 500 baseline subjects
Summarize QC results for each modality
Summarize results for “joint” quality - how many subjects are high quality across all modalities at once?
| p | ||
|---|---|---|
| (N=549) | ||
| qcfail_orientation | ||
| p | ||
|---|---|---|
| (N=523) | ||
| qcfail_background_noise | ||
| p | ||
|---|---|---|
| (N=502) | ||
| NM_QC_Ratings_Z | < 0.001 | |
| p | |||
|---|---|---|---|
| (N=1204) | (N=1340) | ||
| qcfail_orientation | |||
| p | ||||
|---|---|---|---|---|
| (N=2969) | (N=187) | (N=854) | ||
| qcfail_orientation | ||||
| modality | Total | Successes | Failures | Percent_Success | Percent_Failure | failcat |
|---|---|---|---|---|---|---|
| T1w | 523 | 513 | 10 | 98.08795 | 1.9120459 | qchuman_T1w |
| T1w | 523 | 521 | 2 | 99.61759 | 0.3824092 | qcfail_background_noise |
| T1w | 523 | 519 | 4 | 99.23518 | 0.7648184 | qcfail_motion |
| T1w | 523 | 519 | 4 | 99.23518 | 0.7648184 | qcfail_other |
| modality | Total | Successes | Failures | Percent_Success | Percent_Failure | failcat |
|---|---|---|---|---|---|---|
| T2Flair | 549 | 524 | 25 | 95.44627 | 4.5537341 | qchuman_FLAIR |
| T2Flair | 549 | 548 | 1 | 99.81785 | 0.1821494 | qcfail_orientation |
| T2Flair | 549 | 545 | 4 | 99.27140 | 0.7285974 | qcfail_intensity_saturation |
| T2Flair | 549 | 543 | 6 | 98.90710 | 1.0928962 | qcfail_background_noise |
| T2Flair | 549 | 539 | 10 | 98.17851 | 1.8214936 | qcfail_motion |
| T2Flair | 549 | 547 | 2 | 99.63570 | 0.3642987 | qcfail_other |
| modality | Total | Successes | Failures | Percent_Success | Percent_Failure | failcat |
|---|---|---|---|---|---|---|
| DTIb0 | 1204 | 1110 | 94 | 92.19269 | 7.8073090 | qchuman_DTI |
| DTIdwi | 1340 | 1224 | 116 | 91.34328 | 8.6567164 | qchuman_DTI |
| DTIb0 | 1204 | 1188 | 16 | 98.67110 | 1.3289037 | qcfail_orientation |
| DTIdwi | 1340 | 1324 | 16 | 98.80597 | 1.1940299 | qcfail_orientation |
| DTIb0 | 1204 | 1198 | 6 | 99.50166 | 0.4983389 | qcfail_phantom |
| DTIdwi | 1340 | 1334 | 6 | 99.55224 | 0.4477612 | qcfail_phantom |
| DTIb0 | 1204 | 1204 | 0 | 100.00000 | 0.0000000 | qcfail_intensity_saturation |
| DTIdwi | 1340 | 1336 | 4 | 99.70149 | 0.2985075 | qcfail_intensity_saturation |
| DTIb0 | 1204 | 1154 | 50 | 95.84718 | 4.1528239 | qcfail_background_noise |
| DTIdwi | 1340 | 1286 | 54 | 95.97015 | 4.0298507 | qcfail_background_noise |
| DTIb0 | 1204 | 1150 | 54 | 95.51495 | 4.4850498 | qcfail_motion |
| DTIdwi | 1340 | 1270 | 70 | 94.77612 | 5.2238806 | qcfail_motion |
| DTIb0 | 1204 | 1190 | 14 | 98.83721 | 1.1627907 | qcfail_signal_dropout |
| DTIdwi | 1340 | 1318 | 22 | 98.35821 | 1.6417910 | qcfail_signal_dropout |
| DTIb0 | 1204 | 1188 | 16 | 98.67110 | 1.3289037 | qcfail_other |
| DTIdwi | 1340 | 1324 | 16 | 98.80597 | 1.1940299 | qcfail_other |
| modality | Total | Successes | Failures | Percent_Success | Percent_Failure | failcat |
|---|---|---|---|---|---|---|
| rsfMRI | 2969 | 2512 | 457 | 84.60761 | 15.3923880 | qchuman_rsfMRI |
| rsfMRI_LR | 187 | 168 | 19 | 89.83957 | 10.1604278 | qchuman_rsfMRI |
| rsfMRI_RL | 854 | 758 | 96 | 88.75878 | 11.2412178 | qchuman_rsfMRI |
| rsfMRI | 2969 | 2960 | 9 | 99.69687 | 0.3031324 | qcfail_orientation |
| rsfMRI_LR | 187 | 186 | 1 | 99.46524 | 0.5347594 | qcfail_orientation |
| rsfMRI_RL | 854 | 838 | 16 | 98.12646 | 1.8735363 | qcfail_orientation |
| rsfMRI | 2969 | 2955 | 14 | 99.52846 | 0.4715392 | qcfail_phantom |
| rsfMRI_LR | 187 | 187 | 0 | 100.00000 | 0.0000000 | qcfail_phantom |
| rsfMRI_RL | 854 | 854 | 0 | 100.00000 | 0.0000000 | qcfail_phantom |
| rsfMRI | 2969 | 2955 | 14 | 99.52846 | 0.4715392 | qcfail_spacing |
| rsfMRI_LR | 187 | 187 | 0 | 100.00000 | 0.0000000 | qcfail_spacing |
| rsfMRI_RL | 854 | 854 | 0 | 100.00000 | 0.0000000 | qcfail_spacing |
| rsfMRI | 2969 | 2949 | 20 | 99.32637 | 0.6736275 | qcfail_intensity_saturation |
| rsfMRI_LR | 187 | 177 | 10 | 94.65241 | 5.3475936 | qcfail_intensity_saturation |
| rsfMRI_RL | 854 | 822 | 32 | 96.25293 | 3.7470726 | qcfail_intensity_saturation |
| rsfMRI | 2969 | 2388 | 581 | 80.43112 | 19.5688784 | qcfail_motion |
| rsfMRI_LR | 187 | 187 | 0 | 100.00000 | 0.0000000 | qcfail_motion |
| rsfMRI_RL | 854 | 854 | 0 | 100.00000 | 0.0000000 | qcfail_motion |
| rsfMRI | 2969 | 2804 | 165 | 94.44257 | 5.5574267 | qcfail_signal_dropout |
| rsfMRI_LR | 187 | 172 | 15 | 91.97861 | 8.0213904 | qcfail_signal_dropout |
| rsfMRI_RL | 854 | 806 | 48 | 94.37939 | 5.6206089 | qcfail_signal_dropout |
| rsfMRI | 2969 | 2968 | 1 | 99.96632 | 0.0336814 | qcfail_temporal_noise |
| rsfMRI_LR | 187 | 186 | 1 | 99.46524 | 0.5347594 | qcfail_temporal_noise |
| rsfMRI_RL | 854 | 846 | 8 | 99.06323 | 0.9367681 | qcfail_temporal_noise |
| rsfMRI | 2969 | 2957 | 12 | 99.59582 | 0.4041765 | qcfail_insufficient_volumes |
| rsfMRI_LR | 187 | 187 | 0 | 100.00000 | 0.0000000 | qcfail_insufficient_volumes |
| rsfMRI_RL | 854 | 854 | 0 | 100.00000 | 0.0000000 | qcfail_insufficient_volumes |
| rsfMRI | 2969 | 2968 | 1 | 99.96632 | 0.0336814 | qcfail_FOV |
| rsfMRI_LR | 187 | 187 | 0 | 100.00000 | 0.0000000 | qcfail_FOV |
| rsfMRI_RL | 854 | 854 | 0 | 100.00000 | 0.0000000 | qcfail_FOV |
| NM.QC.cat | Total | Successes | Failures | Percent_Success | Percent_Failure | failcat |
|---|---|---|---|---|---|---|
| overall | 478 | 386 | 92 | 80.75314 | 19.246862 | NM_QC_Ratings_Failures |
| z.coord | 478 | 391 | 87 | 81.79916 | 18.200837 | NM_QC_Ratings_Z |
| sn.vol | 478 | 471 | 7 | 98.53556 | 1.464435 | NM_QC_Ratings_SNVol |
| avg.int | 478 | 473 | 5 | 98.95397 | 1.046025 | NM_QC_Ratings_AvgIntensity |
| sd.int | 478 | 473 | 5 | 98.95397 | 1.046025 | NM_QC_Ratings_SDIntensity |
| max.int | 478 | 473 | 5 | 98.95397 | 1.046025 | NM_QC_Ratings_MaxIntensity |
for each, summarize how processing improves the data
rsfMRI is “easy” to see - throw away motion corrupted data
developed methodology specific for PPMI that was informed by this process
automated QC for these modalities for future work
## [1] " <0> BUILD-V <0> BUILD-V <0> BUILD-V <0> BUILD-V <0> "
## [1] "initialDataTerm: -0.372650010297621 <o> mixer: pca <o> E: cca"
## [1] "Iteration: 1 bestEv: -0.0327933507588981 bestIt: 1 CE: -0.0327933507588981"
## [1] "Iteration: 2 bestEv: -0.0630145184881478 bestIt: 2 CE: -0.0630145184881478"
## [1] "Iteration: 3 bestEv: -0.0968209008706581 bestIt: 3 CE: -0.0968209008706581"
## [1] "Iteration: 4 bestEv: -0.0968209008706581 bestIt: 3 CE: -0.0870532158412072"
## [1] "Iteration: 5 bestEv: -0.0968209008706581 bestIt: 3 CE: -0.0869174491918662"
## [1] "Iteration: 6 bestEv: -0.0968209008706581 bestIt: 3 CE: -0.092816820015101"
## [1] "Iteration: 7 bestEv: -0.120174060553191 bestIt: 7 CE: -0.120174060553191"
## [1] "Iteration: 8 bestEv: -0.120174060553191 bestIt: 7 CE: -0.120174060553191"
## [1] "Iteration: 9 bestEv: -0.12581493976007 bestIt: 9 CE: -0.12581493976007"
## [1] "Iteration: 10 bestEv: -0.128566404189215 bestIt: 10 CE: -0.128566404189215"
## [1] "Iteration: 11 bestEv: -0.129545856667216 bestIt: 11 CE: -0.129545856667216"
## [1] "Iteration: 12 bestEv: -0.13209953844886 bestIt: 12 CE: -0.13209953844886"
## [1] "Iteration: 13 bestEv: -0.136764239989398 bestIt: 13 CE: -0.136764239989398"
## [1] "Iteration: 14 bestEv: -0.136764239989398 bestIt: 13 CE: -0.0756392704157992"
## [1] "Iteration: 15 bestEv: -0.136764239989398 bestIt: 13 CE: -0.0801460646909918"
## [1] "Iteration: 16 bestEv: -0.136764239989398 bestIt: 13 CE: -0.0813127215892158"
## [1] "Iteration: 17 bestEv: -0.136764239989398 bestIt: 13 CE: -0.0813127215892158"
## [1] "Iteration: 18 bestEv: -0.136764239989398 bestIt: 13 CE: -0.136546894715855"
## [1] "Iteration: 19 bestEv: -0.139231177436898 bestIt: 19 CE: -0.139231177436898"
## [1] "Iteration: 20 bestEv: -0.147375833711929 bestIt: 20 CE: -0.147375833711929"
## [1] "Iteration: 21 bestEv: -0.147375833711929 bestIt: 20 CE: -0.140165912343255"
## [1] "Iteration: 22 bestEv: -0.147375833711929 bestIt: 20 CE: -0.140165912343255"
## [1] "Iteration: 23 bestEv: -0.147375833711929 bestIt: 20 CE: -0.140165912343255"
## [1] "Iteration: 24 bestEv: -0.147375833711929 bestIt: 20 CE: -0.140165912343255"
## [1] "Iteration: 25 bestEv: -0.147375833711929 bestIt: 20 CE: -0.142603300356657"
## [1] "Iteration: 26 bestEv: -0.147375833711929 bestIt: 20 CE: -0.142603300356657"
## [1] 0.0630929
## [1] 0.0677649
## [1] 0.03006271
## [1] "1 4.66251279271262e-06"
## [1] 0.03932089
## [1] 0.04654327
## [1] 0.02800733
## [1] "1 0.0694299654970291"
## [1] 0.02564484
## [1] 0.09030771
## [1] 0.06288042
## [1] "1 0.107259374372654"
## [1] 0.03935167
## [1] 0.07410274
## [1] 0.04524996
## [1] "1 0.0243461100163986"
## [1] 0.05274071
## [1] 0.09938837
## [1] 0.006772589
## [1] "1 0.00634355576413451"
## [1] 0.0480356
## [1] 0.06075138
## [1] 0.05512847
## [1] "1 0.071096503387464"
## [1] 0.02138776
## [1] 0.08761372
## [1] 0.05981309
## [1] "1 0.0240337878537852"
## [1] 0.02558045
## [1] 0.01475583
## [1] 0.03563923
## [1] "1 0.0213912954696561"
## [1] 0.04264125
## [1] 0.04388821
## [1] 0.02728175
## [1] "1 0.0803078437115961"
## [1] 0.03473762
## [1] 0.0784961
## [1] 0.02471271
## [1] "1 0.141496722736202"
## [1] 0.03666652
## [1] 0.07330025
## [1] 0.04110507
## [1] "1 0.105022161584968"
## [1] 0.07665053
## [1] 0.05541179
## [1] 0.01231193
## [1] "1 0.190776623087218"
## [1] 0.0245232
## [1] 0.05219262
## [1] 0.02625722
## [1] "1 0.624302872606679"
## [1] 0.01198805
## [1] 0.08292536
## [1] 0.008450539
## [1] "1 0.00083372608723358"
## [1] 0.02269778
## [1] 0.06435739
## [1] 0.05076787
## [1] "1 0.0643833172867185"
## [1] 0.03958828
## [1] 0.0702159
## [1] 0.07998912
## [1] "1 0.146026884376684"
## [1] 0.03573261
## [1] 0.06433189
## [1] 0.0185527
## [1] "1 0.0106873200247041"
## [1] 0.02221048
## [1] 0.0930238
## [1] 0.03323806
## [1] "1 0.790196559511591"
## [1] 0.03328113
## [1] 0.06983351
## [1] 0.02989873
## [1] "1 0.0207943312309834"
## [1] 0.03668805
## [1] 0.07834961
## [1] 0.02317041
## [1] "1 0.0295521350585212"
## [1] 0.03173482
## [1] 0.07647341
## [1] 0.05119379
## [1] "1 0.000682459128512673"
## [1] 0.03376814
## [1] 0.09100429
## [1] 0.0563345
## [1] "1 0.0947597558790788"
## [1] 0.03146501
## [1] 0.06099048
## [1] 0.03151784
## [1] "1 0.0223964491684715"
## [1] 0.07344633
## [1] 0.07443348
## [1] 0.01807985
## [1] "1 0.0009147437862368"
## [1] 0.02739198
## [1] 0.04413951
## [1] 0.03004524
## [1] "1 0.0249532674914957"
## [1] 0.03150449
## [1] 0.06983519
## [1] 0.0257677
## [1] "1 0.198556150279165"
## [1] 0.03652763
## [1] 0.05812419
## [1] 0.02950058
## [1] "1 0.0146066682040569"
## [1] 0.05752286
## [1] 0.05845707
## [1] 0.02224718
## [1] "1 0.0196024818996873"
## [1] 0.08553855
## [1] 0.03525787
## [1] 0.05307525
## [1] "1 0.665236161534918"
## [1] 0.04295326
## [1] 0.06219984
## [1] 0.03101205
## [1] "1 0.805840738599311"
## [1] 0.04367623
## [1] 0.03603261
## [1] 0.04932394
## [1] "1 0.0749508019139088"
## [1] 0.03637389
## [1] 0.05323645
## [1] 0.02778571
## [1] "1 0.0448684934285207"
## [1] 0.06125937
## [1] 0.05808785
## [1] 0.02467603
## [1] "1 0.0152548085353898"
## [1] 0.02355944
## [1] 0.09227446
## [1] 0.03216665
## [1] "1 0.775843470055139"
## [1] 0.04717993
## [1] 0.06205458
## [1] 0.02277125
## [1] "1 0.097485275921731"
## [1] 0.06404664
## [1] 0.04678307
## [1] 0.03574624
## [1] "1 0.0690886408416247"
## [1] 0.08644223
## [1] 0.05835425
## [1] 0.04689323
## [1] "1 0.00999000049810171"
## [1] 0.05214455
## [1] 0.09420999
## [1] 0.03095386
## [1] "1 0.514774609975741"
## [1] 0.02561214
## [1] 0.06737521
## [1] 0.06641868
## [1] "1 0.0255923006190755"
## [1] 0.04129444
## [1] 0.04377594
## [1] 0.02938252
## [1] "1 0.00194688486552"
## [1] 0.01765454
## [1] 0.04193728
## [1] 0.02178062
## [1] "1 0.125799252130385"
## [1] 0.02529095
## [1] 0.09921389
## [1] 0.02759971
## [1] "1 0.0267451172679475"
## [1] 0.04072942
## [1] 0.09658272
## [1] 0.0496055
## [1] "1 0.160545179779455"
## [1] 0.02284878
## [1] 0.08965439
## [1] 0.03945563
## [1] "1 0.0541901103364408"
## [1] 0.02106304
## [1] 0.06575828
## [1] 0.005659268
## [1] "1 0.0587620768638725"
## [1] 0.01570877
## [1] 0.07027293
## [1] 0.05540485
## [1] "1 0.0853013080067246"
## [1] 0.02165442
## [1] 0.06689294
## [1] 0.05584299
## [1] "1 0.00559602156321104"
## [1] 0.04891857
## [1] 0.08293585
## [1] 0.02674207
## [1] "1 0.00602111137916511"
## [1] 0.02753027
## [1] 0.08932279
## [1] 0.03674929
## [1] "1 0.0913966257147703"
## [1] 0.03140498
## [1] 0.04357756
## [1] 0.03737233
## [1] "1 0.350222593167011"
## [1] 0.03241135
## [1] 0.0401226
## [1] 0.06252411
## [1] "1 0.108368520216622"
## [1] 0.03677997
## [1] 0.1317392
## [1] 0.009142108
## [1] "1 0.801614903606815"
## [1] 0.0459863
## [1] 0.06341489
## [1] 0.03998678
## [1] "1 0.00438758200079222"
## [1] 0.06326474
## [1] 0.0289275
## [1] 0.0334835
## [1] "1 0.0620506443334716"
## [1] 0.04155094
## [1] 0.03922288
## [1] 0.02816804
## [1] "1 0.0929356483918441"
## [1] 0.04992943
## [1] 0.1033501
## [1] 0.03095999
## [1] "1 0.0400357755810751"
## [1] 0.04161823
## [1] 0.05429088
## [1] 0.05313681
## [1] "1 0.0724555855092227"
## [1] 0.04052251
## [1] 0.08370206
## [1] 0.02893663
## [1] "1 0.402128051312556"
## [1] 0.03919301
## [1] 0.03660509
## [1] 0.03844216
## [1] "1 0.0569052157038563"
## [1] 0.01436859
## [1] 0.01814266
## [1] 0.04979079
## [1] "1 0.0750359938885412"
## [1] 0.04493444
## [1] 0.1125809
## [1] 0.07546835
## [1] "1 0.0549330571836182"
## [1] 0.02603415
## [1] 0.05414639
## [1] 0.06647391
## [1] "1 0.118521595218043"
## [1] 0.04272542
## [1] 0.07975711
## [1] 0.04239525
## [1] "1 0.4452017304738"
## [1] 0.04304212
## [1] 0.05476584
## [1] 0.03708388
## [1] "1 0.00330776124436703"
## [1] 0.03181756
## [1] 0.03243388
## [1] 0.04640582
## [1] "1 0.948508253740791"
## [1] 0.06655578
## [1] 0.06839427
## [1] 0.01590732
## [1] "1 0.72330040215693"
## [1] 0.0154856
## [1] 0.02257197
## [1] 0.02519842
## [1] "1 0.071921885311281"
## [1] 0.01837715
## [1] 0.05024716
## [1] 0.04997093
## [1] "1 0.617639396084375"
## [1] 0.0605691
## [1] 0.116757
## [1] 0.07546761
## [1] "1 0.268881609018105"
## [1] 0.06633092
## [1] 0.08038766
## [1] 0.08946624
## [1] "1 0.0635555173143578"
## [1] 0.06078885
## [1] 0.07235237
## [1] 0.03320166
## [1] "1 0.0772495056408294"
## [1] 0.06277849
## [1] 0.06907499
## [1] 0.02451351
## [1] "1 0.652967102154168"
## [1] 0.03819196
## [1] 0.08308102
## [1] 0.05618294
## [1] "1 0.102773856216944"
## [1] 0.02484044
## [1] 0.07820951
## [1] 0.008306815
## [1] "1 0.0832899537153189"
## [1] 0.03610768
## [1] 0.06570733
## [1] 0.04691317
## [1] "1 0.051203573311852"
## [1] 0.04241024
## [1] 0.07551714
## [1] 0.02287316
## [1] "1 0.0126698285088394"
## [1] 0.03192555
## [1] 0.04344844
## [1] 0.03156755
## [1] "1 0.00956121475379167"
## [1] 0.02104856
## [1] 0.0649079
## [1] 0.0180208
## [1] "1 0.16148777507621"
## [1] 0.01960253
## [1] 0.04306293
## [1] 0.04180044
## [1] "1 0.0503950715107181"
## [1] 0.04204938
## [1] 0.06589457
## [1] 0.02142847
## [1] "1 0.183010415225421"
## [1] 0.04359833
## [1] 0.06302444
## [1] 0.03822532
## [1] "1 0.174111623702899"
## [1] 0.04065948
## [1] 0.05185999
## [1] 0.0346206
## [1] "1 0.0199213036312554"
## [1] 0.03220299
## [1] 0.06735252
## [1] 0.04200663
## [1] "1 0.274081720575057"
## [1] 0.01462172
## [1] 0.03904959
## [1] 0.01407211
## [1] "1 0.0467825633887689"
## [1] 0.06881481
## [1] 0.08099738
## [1] 0.04328546
## [1] "1 0.122582259172455"
## [1] 0.02522416
## [1] 0.04433409
## [1] 0.05660003
## [1] "1 0.590313197992923"
## [1] 0.04195973
## [1] 0.04705147
## [1] 0.01997471
## [1] "1 0.47715636518013"
## [1] 0.03639515
## [1] 0.04717104
## [1] 0.03548255
## [1] "1 0.0128221632649177"
## [1] 0.03828957
## [1] 0.0419498
## [1] 0.0197351
## [1] "1 0.0210430656699457"
## [1] 0.03973611
## [1] 0.06136413
## [1] 0.0256119
## [1] "1 0.249424783537397"
## [1] 0.04674051
## [1] 0.06373942
## [1] 0.01353346
## [1] "1 0.20309331060424"
## [1] 0.05356098
## [1] 0.09452549
## [1] 0.04410354
## [1] "1 0.56699252753519"
## [1] 0.02876336
## [1] 0.09104336
## [1] 0.02855228
## [1] "1 0.0442650152963898"
## [1] 0.04259282
## [1] 0.09626909
## [1] 0.0229427
## [1] "1 0.0254021432991888"
## [1] 0.03816867
## [1] 0.04962869
## [1] 0.03666615
## [1] "1 0.0330042481634965"
## [1] 0.005864365
## [1] 0.03390455
## [1] 0.02994949
## [1] "1 0.0415025307632421"
## [1] 0.03045308
## [1] 0.0453398
## [1] 0.03063926
## [1] "1 0.0485317867560451"
## [1] 0.02098169
## [1] 0.04667136
## [1] 0.0205433
## [1] "1 0.221197703317231"
## [1] 0.02825009
## [1] 0.08395947
## [1] 0.0163654
## [1] "1 0.0331833196804612"
## [1] 0.06827637
## [1] 0.08456064
## [1] 0.06209281
## [1] "1 0.113786966265866"
## [1] 0.03580802
## [1] 0.07874386
## [1] 0.01894777
## [1] "1 0.717608326962784"
## [1] 0.05758521
## [1] 0.04530091
## [1] 0.01051303
## [1] "1 0.18398831889247"
## [1] 0.02186671
## [1] 0.04610322
## [1] 0.0374948
## [1] "1 0.0463069996687485"
## [1] 0.04641669
## [1] 0.04972749
## [1] 0.02195146
## [1] "1 0.14269496939746"
## [1] 0.06868643
## [1] 0.06292221
## [1] 0.05070293
## [1] "1 0.0185477389632554"
## [1] 0.05564
## [1] 0.08068596
## [1] 0.05502952
## [1] "1 0.587176282924374"
## [1] 0.04066368
## [1] 0.03527615
## [1] 0.03610686
## [1] "1 0.00132705617918233"
## [1] 0.05889093
## [1] 0.05160893
## [1] 0.06224246
## [1] "1 0.0670994235322177"
## [1] 0.02493003
## [1] 0.07788846
## [1] 0.01221224
## [1] "1 0.0242276829624712"
## [1] 0.02786918
## [1] 0.06390757
## [1] 0.02708857
## [1] "1 0.087813684505013"
## [1] 0.03467518
## [1] 0.07589571
## [1] 0.06250838
## [1] "1 0.31586539187297"
## [1] 0.05978308
## [1] 0.06222272
## [1] 0.02904438
## [1] "1 0.313657587625811"
## [1] 0.0309592
## [1] 0.04773233
## [1] 0.0354711
## [1] "1 0.0136265568514024"
## [1] 0.04713433
## [1] 0.05349283
## [1] 0.04030734
## [1] "1 0.559946659467991"
## [1] 0.06111391
## [1] 0.05720246
## [1] 0.06445698
## [1] "1 0.0557059143677796"
## [1] 0.02169783
## [1] 0.09261248
## [1] 0.03790334
## [1] "1 0.161030241134552"
## [1] 0.05513028
## [1] 0.06287655
## [1] 0.02330846
## [1] "1 0.141613043712074"
## [1] 0.04852659
## [1] 0.06958726
## [1] 0.02339817
## [1] "1 0.387182361652895"
## [1] 0.05788892
## [1] 0.0843384
## [1] 0.0216129
## [1] "1 0.0374318376245641"
## [1] 0.03105342
## [1] 0.03727803
## [1] 0.05677604
## [1] "1 0.0382186072401221"
## [1] 0.02512108
## [1] 0.07223662
## [1] 0.03961764
## [1] "1 0.0343678488559571"
## [1] 0.01854065
## [1] 0.04150163
## [1] 0.03064332
## [1] "1 0.0366600712226003"
## [1] 0.047196
## [1] 0.06133113
## [1] 0.01940903
## [1] "1 0.556195431398713"
## [1] 0.05281333
## [1] 0.04062543
## [1] 0.03744118
## [1] "1 0.161613250646088"
## [1] 0.03855365
## [1] 0.02801116
## [1] 0.02297174
## [1] "1 0.192306688094241"
## [1] 0.03333551
## [1] 0.0550211
## [1] 0.04167407
## [1] "1 0.725865168670112"
## [1] 0.06487669
## [1] 0.05997242
## [1] 0.00824551
## [1] "1 0.0595112379510438"
## [1] 0.0182962
## [1] 0.03256486
## [1] 0.04489314
## [1] "1 0.121290868638493"
## [1] 0.03392788
## [1] 0.06008074
## [1] 0.06343774
## [1] "1 0.00545932036091891"
## [1] 0.06208573
## [1] 0.0688373
## [1] 0.05679632
## [1] "1 0.699649464625945"
## [1] 0.009284302
## [1] 0.04557386
## [1] 0.01069078
## [1] "1 0.00652772077655523"
## [1] 0.02575567
## [1] 0.07885447
## [1] 0.03236572
## [1] "1 0.173394371085031"
## [1] 0.03213179
## [1] 0.07631367
## [1] 0.02437731
## [1] "1 0.251820688673548"
## [1] 0.04118437
## [1] 0.02579297
## [1] 0.03588077
## [1] "1 0.163872144705226"
## [1] 0.0558201
## [1] 0.0521762
## [1] 0.028283
## [1] "1 0.356186558503165"
## [1] 0.0109955
## [1] 0.06303201
## [1] 0.02731521
## [1] "1 0.138329646229258"
## [1] 0.04120813
## [1] 0.0495756
## [1] 0.02899944
## [1] "1 0.0528586571631784"
## [1] 0.05082004
## [1] 0.07823377
## [1] 0.04619435
## [1] "1 0.0170658100592988"
## [1] 0.00858112
## [1] 0.004596902
## [1] 0.009437389
## [1] "1 0.100905524126079"
## [1] 0.05102408
## [1] 0.08837339
## [1] 0.0531868
## [1] "1 0.0171382945300328"
## [1] 0.03291919
## [1] 0.0553402
## [1] 0.02796184
## [1] "1 0.337177807747296"
## [1] 0.05459665
## [1] 0.03900464
## [1] 0.03572071
## [1] "1 0.153461346358315"
## [1] 0.04726743
## [1] 0.06830262
## [1] 0.03259438
## [1] "1 0.415804645433589"
## [1] 0.0177201
## [1] 0.06061726
## [1] 0.02973647
## [1] "1 0.16571572102884"
## [1] 0.04017365
## [1] 0.09883411
## [1] 0.04943956
## [1] "1 0.16844120146565"
## [1] 0.03433618
## [1] 0.07503754
## [1] 0.03671217
## [1] "1 0.404340637721021"
## [1] 0.0340711
## [1] 0.04898522
## [1] 0.04845004
## [1] "1 0.0736905253475063"
## [1] 0.03212837
## [1] 0.07287336
## [1] 0.06014556
## [1] "1 0.604708481761244"
## [1] 0.06430525
## [1] 0.07926983
## [1] 0.0247576
## [1] "1 0.144684532290054"
## [1] 0.04720213
## [1] 0.09778086
## [1] 0.04477515
## [1] "1 0.0822021566562165"
## [1] 0.01946235
## [1] 0.04567663
## [1] 0.05489625
## [1] "1 0.00977641272576385"
## [1] 0.04997235
## [1] 0.08113583
## [1] 0.04181768
## [1] "1 0.227776216761175"
## [1] 0.05166893
## [1] 0.06268007
## [1] 0.04021258
## [1] "1 0.050305054255458"
## [1] 0.04414744
## [1] 0.05509166
## [1] 0.01084672
## [1] "1 0.222739058256784"
## [1] 0.01237579
## [1] 0.005355435
## [1] 0.01219065
## [1] "1 0.1687876333614"
## [1] 0.03688722
## [1] 0.1335837
## [1] 0.02755578
## [1] "1 0.485784902815789"
## [1] 0.07897884
## [1] 0.03711085
## [1] 0.02465954
## [1] "1 0.0123230111060842"
## [1] 0.0448215
## [1] 0.07793249
## [1] 0.01497113
## [1] "1 0.00453776382408584"
## [1] 0.02108004
## [1] 0.07203351
## [1] 0.02664318
## [1] "1 0.360589701654667"
## [1] 0.03816867
## [1] 0.04962869
## [1] 0.03666615
## [1] "1 0.0330042481634965"
## [1] 0.06027731
## [1] 0.08355711
## [1] 0.05492718
## [1] "1 0.539366919625769"
## [1] 0.03788948
## [1] 0.02895126
## [1] 0.03721508
## [1] "1 0.00432227230294628"
## [1] 0.04841777
## [1] 0.03165477
## [1] 0.033069
## [1] "1 0.0746250184154798"
## [1] 0.0630172
## [1] 0.0517884
## [1] 0.05298815
## [1] "1 0.0829313235481923"
## [1] 0.03965592
## [1] 0.06163888
## [1] 0.03030191
## [1] "1 0.358820793829194"
## [1] 0.009545613
## [1] 0.07979472
## [1] 0.01954466
## [1] "1 0.0585191639721679"
## [1] 0.06868643
## [1] 0.06292221
## [1] 0.05070293
## [1] "1 0.0185477389632554"
## [1] 0.07132661
## [1] 0.07377478
## [1] 0.02631914
## [1] "1 0.527004824669165"
## [1] 0.03441058
## [1] 0.09505394
## [1] 0.01677284
## [1] "1 0.0488496378477023"
## [1] 0.01610217
## [1] 0.07306119
## [1] 0.02858364
## [1] "1 0.0157040997167592"
## [1] 0.06325705
## [1] 0.1084706
## [1] 0.04432835
## [1] "1 0.0194102902393112"
## [1] 0.04399229
## [1] 0.07010635
## [1] 0.03007796
## [1] "1 0.307944925682457"
## [1] 0.05415916
## [1] 0.04006834
## [1] 0.02738588
## [1] "1 0.105228511657066"
## [1] 0.04336709
## [1] 0.06706398
## [1] 0.03467856
## [1] "1 0.0278014286085451"
## [1] 0.04909176
## [1] 0.09420704
## [1] 0.02671771
## [1] "1 0.152552318255103"
## [1] 0.04756534
## [1] 0.05353695
## [1] 0.01422154
## [1] "1 0.0159567027794037"
## [1] 0.04715246
## [1] 0.05558576
## [1] 0.02785374
## [1] "1 0.0467562147285988"
## [1] 0.04532046
## [1] 0.1340682
## [1] 0.02261944
## [1] "1 0.0205892991899342"
## [1] 0.04184438
## [1] 0.02770159
## [1] 0.03000734
## [1] "1 0.254904107118623"
## [1] 0.02094793
## [1] 0.05395701
## [1] 0.02896039
## [1] "1 0.662727805310266"
## [1] 0.02677782
## [1] 0.04816317
## [1] 0.02323123
## [1] "1 0.0720307668928356"
## [1] 0.04428827
## [1] 0.04965497
## [1] 0.01669672
## [1] "1 0.0597032323785957"
## [1] 0.06593947
## [1] 0.08173966
## [1] 0.02927645
## [1] "1 0.0102231872300007"
## [1] 0.05834271
## [1] 0.08402735
## [1] 0.03180263
## [1] "1 0.107761533416821"
## [1] 0.03695462
## [1] 0.05395324
## [1] 0.03501445
## [1] "1 0.0284423480687222"
## [1] 0.02674508
## [1] 0.05297481
## [1] 0.04305638
## [1] "1 0.0152850993354577"
## [1] 0.06800031
## [1] 0.07924679
## [1] 0.0293476
## [1] "1 0.137354527720979"
## [1] 0.04909237
## [1] 0.03752832
## [1] 0.02931085
## [1] "1 0.0163764661782822"
## [1] 0.03837174
## [1] 0.09515757
## [1] 0.01964217
## [1] "1 0.16916548982543"
## [1] 0.03735237
## [1] 0.06716148
## [1] 0.03686288
## [1] "1 0.0123505264571607"
## [1] 0.05561619
## [1] 0.1035607
## [1] 0.0543239
## [1] "1 0.678139952645981"
## [1] 0.04208442
## [1] 0.05712025
## [1] 0.01417198
## [1] "1 0.83175904487243"
## [1] 0.08236342
## [1] 0.09283336
## [1] 0.03569142
## [1] "1 0.774343043103551"
## [1] 0.04271123
## [1] 0.06501948
## [1] 0.04764576
## [1] "1 0.118975711224574"
## [1] 0.05231901
## [1] 0.04154095
## [1] 0.0189346
## [1] "1 0.504226123155032"
## [1] 0.04351042
## [1] 0.07920088
## [1] 0.04696099
## [1] "1 0.0498942977024452"
## [1] 0.02987292
## [1] 0.05440064
## [1] 0.05568823
## [1] "1 0.0205641883157072"
## [1] 0.03518637
## [1] 0.06482327
## [1] 0.01457954
## [1] "1 0.31120838382253"
## [1] 0.03586346
## [1] 0.06525843
## [1] 0.04924828
## [1] "1 0.0848066713776339"
## [1] 0.05856011
## [1] 0.05774023
## [1] 0.04834239
## [1] "1 0.140705367548224"
## [1] 0.04093962
## [1] 0.1164789
## [1] 0.05582157
## [1] "1 0.00227655973751552"
| permstate | anv | pvalt1 | pvalrs | pvaldt |
|---|---|---|---|---|
| original | 0.0000047 | 0.0011691 | 0.0079420 | 0.0003504 |
| perm | 0.0694300 | 0.0802097 | 0.2667030 | 0.0798390 |
| perm | 0.1072594 | 0.8680470 | 0.3132921 | 0.0222560 |
| perm | 0.0243461 | 0.1413078 | 0.5740906 | 0.0056556 |
| perm | 0.0063436 | 0.1798453 | 0.0449181 | 0.0167461 |
| perm | 0.0710965 | 0.1128826 | 0.9834749 | 0.0242626 |
| perm | 0.0240338 | 0.7456317 | 0.0665245 | 0.0125568 |
| perm | 0.0213913 | 0.2119596 | 0.3868857 | 0.0027174 |
| perm | 0.0803078 | 0.0382954 | 0.6368228 | 0.0916118 |
| perm | 0.1414967 | 0.2344531 | 0.5574916 | 0.0469212 |
| perm | 0.1050222 | 0.2381957 | 0.0667943 | 0.0745796 |
| perm | 0.1907766 | 0.4492391 | 0.3834974 | 0.0571751 |
| perm | 0.6243029 | 0.8945569 | 0.3675913 | 0.3459025 |
| perm | 0.0008337 | 0.0580824 | 0.1650137 | 0.0005346 |
| perm | 0.0643833 | 0.1016979 | 0.4763794 | 0.0315559 |
| perm | 0.1460269 | 0.6870602 | 0.0925345 | 0.1338071 |
| perm | 0.0106873 | 0.0443860 | 0.7994354 | 0.0035969 |
| perm | 0.7901966 | 0.4075784 | 0.8025325 | 0.4682103 |
| perm | 0.0207943 | 0.5688331 | 0.6109862 | 0.0022387 |
| perm | 0.0295521 | 0.4142411 | 0.0774829 | 0.0216190 |
| perm | 0.0006825 | 0.2971841 | 0.1911848 | 0.0001482 |
| perm | 0.0947598 | 0.5302543 | 0.2436291 | 0.0236403 |
| perm | 0.0223964 | 0.0634172 | 0.5029030 | 0.0094847 |
| perm | 0.0009147 | 0.0705353 | 0.1314502 | 0.0007966 |
| perm | 0.0249533 | 0.1737466 | 0.2916859 | 0.0069855 |
| perm | 0.1985562 | 0.8680367 | 0.6774136 | 0.0325414 |
| perm | 0.0146067 | 0.0710195 | 0.1551414 | 0.0233536 |
| perm | 0.0196025 | 0.9963803 | 0.2418433 | 0.0035880 |
| perm | 0.6652362 | 0.3755459 | 0.8568189 | 0.3702365 |
| perm | 0.8058407 | 0.5812390 | 0.8286369 | 0.3912588 |
| perm | 0.0749508 | 0.8351105 | 0.0900523 | 0.0392643 |
| perm | 0.0448685 | 0.4374698 | 0.1231911 | 0.0240846 |
| perm | 0.0152548 | 0.0929824 | 0.3275293 | 0.0043716 |
| perm | 0.7758435 | 0.4446225 | 0.9288389 | 0.4930635 |
| perm | 0.0974853 | 0.0960748 | 0.9238726 | 0.0456830 |
| perm | 0.0690886 | 0.1556651 | 0.0452718 | 0.2670161 |
| perm | 0.0099900 | 0.0439791 | 0.5721626 | 0.0068389 |
| perm | 0.5147746 | 0.6636876 | 0.3875821 | 0.2499281 |
| perm | 0.0255923 | 0.0438977 | 0.4616903 | 0.0243061 |
| perm | 0.0019469 | 0.0264891 | 0.3289980 | 0.0008280 |
| perm | 0.1257993 | 0.0943011 | 0.2675880 | 0.0837799 |
| perm | 0.0267451 | 0.1978715 | 0.8388439 | 0.0075719 |
| perm | 0.1605452 | 0.4487515 | 0.3234861 | 0.0607759 |
| perm | 0.0541901 | 0.3548631 | 0.4592467 | 0.0106347 |
| perm | 0.0587621 | 0.8426991 | 0.0957213 | 0.0259452 |
| perm | 0.0853013 | 0.1784287 | 0.6893937 | 0.0217681 |
| perm | 0.0055960 | 0.0930459 | 0.7845574 | 0.0008070 |
| perm | 0.0060211 | 0.1623494 | 0.1943731 | 0.0022386 |
| perm | 0.0913966 | 0.0551502 | 0.3636889 | 0.0991148 |
| perm | 0.3502226 | 0.3775817 | 0.5509221 | 0.1273804 |
| perm | 0.1083685 | 0.0953034 | 0.8018574 | 0.0684454 |
| perm | 0.8016149 | 0.4136429 | 0.7170129 | 0.7642400 |
| perm | 0.0043876 | 0.0384879 | 0.1571281 | 0.0032148 |
| perm | 0.0620506 | 0.6822488 | 0.5852356 | 0.0091929 |
| perm | 0.0929356 | 0.4696312 | 0.0529441 | 0.1507264 |
| perm | 0.0400358 | 0.1018730 | 0.5759020 | 0.0296177 |
| perm | 0.0724556 | 0.0862836 | 0.3388286 | 0.0255312 |
| perm | 0.4021281 | 0.4322164 | 0.7817934 | 0.1310847 |
| perm | 0.0569052 | 0.0540474 | 0.5177170 | 0.0407529 |
| perm | 0.0750360 | 0.8257215 | 0.2711839 | 0.0257086 |
| perm | 0.0549331 | 0.7217294 | 0.2921537 | 0.0127905 |
| perm | 0.1185216 | 0.2418733 | 0.2809390 | 0.0494120 |
| perm | 0.4452017 | 0.9142723 | 0.6120951 | 0.1194419 |
| perm | 0.0033078 | 0.1132300 | 0.5610602 | 0.0006265 |
| perm | 0.9485083 | 0.6890804 | 0.8912457 | 0.6391186 |
| perm | 0.7233004 | 0.9193576 | 0.5098847 | 0.3335210 |
| perm | 0.0719219 | 0.1500068 | 0.1576127 | 0.0674096 |
| perm | 0.6176394 | 0.1902574 | 0.8471342 | 0.9370318 |
| perm | 0.2688816 | 0.1490747 | 0.7415507 | 0.1871537 |
| perm | 0.0635555 | 0.7984197 | 0.8479511 | 0.0088242 |
| perm | 0.0772495 | 0.1216349 | 0.2260567 | 0.0770116 |
| perm | 0.6529671 | 0.2899924 | 0.8405496 | 0.4115109 |
| perm | 0.1027739 | 0.3605125 | 0.6181047 | 0.0164950 |
| perm | 0.0832900 | 0.0961793 | 0.2305333 | 0.0550834 |
| perm | 0.0512036 | 0.1351332 | 0.5282547 | 0.0126801 |
| perm | 0.0126698 | 0.0508589 | 0.9829042 | 0.0045834 |
| perm | 0.0095612 | 0.2416418 | 0.5863402 | 0.0011179 |
| perm | 0.1614878 | 0.2291609 | 0.6379811 | 0.0405266 |
| perm | 0.0503951 | 0.7148919 | 0.1093369 | 0.0206718 |
| perm | 0.1830104 | 0.2935424 | 0.9890428 | 0.0480269 |
| perm | 0.1741116 | 0.5026333 | 0.2962551 | 0.0875897 |
| perm | 0.0199213 | 0.0254064 | 0.5499397 | 0.0169188 |
| perm | 0.2740817 | 0.4565420 | 0.5806345 | 0.0688302 |
| perm | 0.0467826 | 0.1513901 | 0.2377942 | 0.0209253 |
| perm | 0.1225823 | 0.3535417 | 0.6136291 | 0.0382534 |
| perm | 0.5903132 | 0.1871730 | 0.7036283 | 0.7227901 |
| perm | 0.4771564 | 0.1285610 | 0.7862284 | 0.6632872 |
| perm | 0.0128222 | 0.0971413 | 0.6561062 | 0.0027370 |
| perm | 0.0210431 | 0.0960209 | 0.2726164 | 0.0109766 |
| perm | 0.2494248 | 0.6314261 | 0.1392429 | 0.1658396 |
| perm | 0.2030933 | 0.2016776 | 0.3892755 | 0.0951724 |
| perm | 0.5669925 | 0.9017372 | 0.4840948 | 0.2259073 |
| perm | 0.0442650 | 0.3834788 | 0.6761647 | 0.0061651 |
| perm | 0.0254021 | 0.1158664 | 0.1227671 | 0.0191241 |
| perm | 0.0330042 | 0.4537452 | 0.0814735 | 0.0292072 |
| perm | 0.0415025 | 0.0616540 | 0.3070899 | 0.0174161 |
| perm | 0.0485318 | 0.3644509 | 0.9941358 | 0.0065105 |
| perm | 0.2211977 | 0.2176258 | 0.2491716 | 0.1556025 |
| perm | 0.0331833 | 0.5037781 | 0.6604987 | 0.0049106 |
| perm | 0.1137870 | 0.4380586 | 0.3557650 | 0.0372716 |
| perm | 0.7176083 | 0.8634680 | 0.5513603 | 0.3816730 |
| perm | 0.1839883 | 0.2040618 | 0.5379042 | 0.0898527 |
| perm | 0.0463070 | 0.6087771 | 0.4922447 | 0.0069033 |
| perm | 0.1426950 | 0.1552440 | 0.1620237 | 0.1751514 |
| perm | 0.0185477 | 0.0836147 | 0.4272136 | 0.0105034 |
| perm | 0.5871763 | 0.8943092 | 0.5416989 | 0.1892451 |
| perm | 0.0013271 | 0.3020791 | 0.8494149 | 0.0000920 |
| perm | 0.0670994 | 0.0475736 | 0.9978720 | 0.0432202 |
| perm | 0.0242277 | 0.0291041 | 0.5129707 | 0.0125404 |
| perm | 0.0878137 | 0.1413292 | 0.6113380 | 0.0703612 |
| perm | 0.3158654 | 0.3058350 | 0.6868090 | 0.1166325 |
| perm | 0.3136576 | 0.2328416 | 0.9915143 | 0.1570859 |
| perm | 0.0136266 | 0.0582303 | 0.3098123 | 0.0096531 |
| perm | 0.5599467 | 0.3081697 | 0.8486026 | 0.3014523 |
| perm | 0.0557059 | 0.0293154 | 0.4986820 | 0.0868731 |
| perm | 0.1610302 | 0.1703099 | 0.4003211 | 0.0743655 |
| perm | 0.1416130 | 0.6775072 | 0.0209411 | 0.8856745 |
| perm | 0.3871824 | 0.1356557 | 0.4107656 | 0.9508726 |
| perm | 0.0374318 | 0.0214268 | 0.2416790 | 0.1116836 |
| perm | 0.0382186 | 0.0605358 | 0.8970215 | 0.0143635 |
| perm | 0.0343678 | 0.3350868 | 0.1625251 | 0.0175428 |
| perm | 0.0366601 | 0.0619787 | 0.4672351 | 0.0130751 |
| perm | 0.5561954 | 0.3095874 | 0.7271770 | 0.3232198 |
| perm | 0.1616133 | 0.1152372 | 0.8813898 | 0.0938115 |
| perm | 0.1923067 | 0.3504283 | 0.5276240 | 0.0547172 |
| perm | 0.7258652 | 0.5943323 | 0.3963854 | 0.4515286 |
| perm | 0.0595112 | 0.4495713 | 0.5696871 | 0.0079844 |
| perm | 0.1212909 | 0.4557049 | 0.8296672 | 0.0188269 |
| perm | 0.0054593 | 0.1796309 | 0.7062818 | 0.0005414 |
| perm | 0.6996495 | 0.6201654 | 0.3274373 | 0.5327888 |
| perm | 0.0065277 | 0.0627990 | 0.9455332 | 0.0010927 |
| perm | 0.1733944 | 0.1267778 | 0.1533224 | 0.3927794 |
| perm | 0.2518207 | 0.2694057 | 0.1324061 | 0.6773695 |
| perm | 0.1638721 | 0.0568458 | 0.9528489 | 0.1787965 |
| perm | 0.3561866 | 0.2153563 | 0.5857127 | 0.1960760 |
| perm | 0.1383296 | 0.1902167 | 0.8546937 | 0.0431959 |
| perm | 0.0528587 | 0.7289491 | 0.0770520 | 0.0518193 |
| perm | 0.0170658 | 0.3383117 | 0.0405216 | 0.0082595 |
| perm | 0.1009055 | 0.0896738 | 0.7441577 | 0.0332771 |
| perm | 0.0171383 | 0.0307829 | 0.3841242 | 0.0189550 |
| perm | 0.3371778 | 0.0902334 | 0.5264933 | 0.5428351 |
| perm | 0.1534613 | 0.6960775 | 0.5423815 | 0.0240024 |
| perm | 0.4158046 | 0.4257152 | 0.4716179 | 0.1704779 |
| perm | 0.1657157 | 0.1687814 | 0.8978533 | 0.0513860 |
| perm | 0.1684412 | 0.0979562 | 0.8377187 | 0.1560366 |
| perm | 0.4043406 | 0.8597943 | 0.0914698 | 0.9780985 |
| perm | 0.0736905 | 0.0836189 | 0.6333398 | 0.0237988 |
| perm | 0.6047085 | 0.3721159 | 0.9754145 | 0.3108546 |
| perm | 0.1446845 | 0.1439272 | 0.7528806 | 0.0702540 |
| perm | 0.0822022 | 0.2586044 | 0.6378371 | 0.0134438 |
| perm | 0.0097764 | 0.3144236 | 0.3216937 | 0.0017270 |
| perm | 0.2277762 | 0.3717628 | 0.2783915 | 0.0942077 |
| perm | 0.0503051 | 0.1145443 | 0.1845273 | 0.0245335 |
| perm | 0.2227391 | 0.0792579 | 0.2357078 | 0.6285385 |
| perm | 0.1687876 | 0.1467620 | 0.8206785 | 0.0544414 |
| perm | 0.4857849 | 0.3336680 | 0.3264938 | 0.3102242 |
| perm | 0.0123230 | 0.1604735 | 0.2419496 | 0.0067785 |
| perm | 0.0045378 | 0.1040823 | 0.0241557 | 0.0181515 |
| perm | 0.3605897 | 0.1477988 | 0.5553645 | 0.2611679 |
| perm | 0.0330042 | 0.4537452 | 0.0814735 | 0.0292072 |
| perm | 0.5393669 | 0.1832187 | 0.5696622 | 0.7826845 |
| perm | 0.0043223 | 0.0750257 | 0.2081106 | 0.0025184 |
| perm | 0.0746250 | 0.0246266 | 0.1072645 | 0.8401808 |
| perm | 0.0829313 | 0.5088470 | 0.2901157 | 0.0203564 |
| perm | 0.3588208 | 0.2545915 | 0.6736570 | 0.2056361 |
| perm | 0.0585192 | 0.0684210 | 0.4396406 | 0.0199962 |
| perm | 0.0185477 | 0.0836147 | 0.4272136 | 0.0105034 |
| perm | 0.5270048 | 0.4450949 | 0.7424349 | 0.2029845 |
| perm | 0.0488496 | 0.2995701 | 0.4118366 | 0.0122171 |
| perm | 0.0157041 | 0.8232076 | 0.5981375 | 0.0015439 |
| perm | 0.0194103 | 0.7081408 | 0.0438179 | 0.0107465 |
| perm | 0.3079449 | 0.2383637 | 0.9796426 | 0.1557321 |
| perm | 0.1052285 | 0.0934900 | 0.8609546 | 0.0443603 |
| perm | 0.0278014 | 0.0145153 | 0.7033555 | 0.0278191 |
| perm | 0.1525523 | 0.4403063 | 0.4325935 | 0.0339006 |
| perm | 0.0159567 | 0.1972291 | 0.4869382 | 0.0047655 |
| perm | 0.0467562 | 0.0667625 | 0.1584228 | 0.0580237 |
| perm | 0.0205893 | 0.0667067 | 0.8420439 | 0.0037786 |
| perm | 0.2549041 | 0.1976972 | 0.6932001 | 0.1738760 |
| perm | 0.6627278 | 0.8070727 | 0.8688831 | 0.2337125 |
| perm | 0.0720308 | 0.5917468 | 0.8213298 | 0.0107102 |
| perm | 0.0597032 | 0.0556809 | 0.7055130 | 0.0486853 |
| perm | 0.0102232 | 0.0347012 | 0.9613956 | 0.0039314 |
| perm | 0.1077615 | 0.5833151 | 0.7243364 | 0.0154177 |
| perm | 0.0284423 | 0.1281072 | 0.6674129 | 0.0087022 |
| perm | 0.0152851 | 0.1267394 | 0.1038448 | 0.0071859 |
| perm | 0.1373545 | 0.4431139 | 0.4776568 | 0.0433358 |
| perm | 0.0163765 | 0.0282572 | 0.2131662 | 0.0305710 |
| perm | 0.1691655 | 0.6417073 | 0.2249495 | 0.0559773 |
| perm | 0.0123505 | 0.0658297 | 0.1835070 | 0.0121501 |
| perm | 0.6781400 | 0.8451845 | 0.6806925 | 0.2457390 |
| perm | 0.8317590 | 0.8303224 | 0.3975545 | 0.7816628 |
| perm | 0.7743430 | 0.5959346 | 0.5265119 | 0.5092359 |
| perm | 0.1189757 | 0.1273301 | 0.5804916 | 0.0392976 |
| perm | 0.5042261 | 0.2928782 | 0.7599939 | 0.2445645 |
| perm | 0.0498943 | 0.0951265 | 0.0336678 | 0.2589517 |
| perm | 0.0205642 | 0.3440710 | 0.3760162 | 0.0047590 |
| perm | 0.3112084 | 0.1026529 | 0.6954546 | 0.2563652 |
| perm | 0.0848067 | 0.9943544 | 0.2818580 | 0.0249202 |
| perm | 0.1407054 | 0.3576636 | 0.4492610 | 0.0382356 |
| perm | 0.0022766 | 0.4589839 | 0.4750495 | 0.0002665 |
## Warning: Using `bins = 30` by default. Pick better value with the argument
## `bins`.